Developing Component Scores from Natural Language Processing Tools to Assess Human Ratings of Essay Quality
نویسندگان
چکیده
This study explores correlations between human ratings of essay quality and component scores based on similar natural language processing indices and weighted through a principal component analysis. The results demonstrate that such component scores show small to large effects with human ratings and thus may be suitable to providing both summative and formative feedback in an automatic writing evaluation systems such as those found in Writing-Pal.
منابع مشابه
Predicting Misalignment Between Teachers' and Students' Essay Scores Using Natural Language Processing Tools
We investigated linguistic factors that relate to misalignment between students’ and teachers’ ratings of essay quality. Students (n = 126) wrote essays and rated the quality of their work. Teachers then provided their own ratings of the essays. Results revealed that students who were less accurate in their self-assessments produced essays that were more causal, contained less meaningful words,...
متن کاملNatural language processing in an intelligent writing strategy tutoring system.
The Writing Pal is an intelligent tutoring system that provides writing strategy training. A large part of its artificial intelligence resides in the natural language processing algorithms to assess essay quality and guide feedback to students. Because writing is often highly nuanced and subjective, the development of these algorithms must consider a broad array of linguistic, rhetorical, and c...
متن کاملIncorporating learning characteristics into automatic essay scoring models: What individual differences and linguistic features tell us about writing quality
This study investigates a novel approach to automatically assessing essay quality that combines natural language processing approaches that assess text features with approaches that assess individual differences in writers such as demographic information, standardized test scores, and survey results. The results demonstrate that combining text features and individual differences increases the a...
متن کاملBenefits of Modularity in an Automated Essay Scoring System
E-rater is an operational automated essay scoring application that combines several NLP tools for the purpose of identifying linguistic features in essay responses to assess the quality of the text. The application currently identifies a variety of syntactic, discourse, and topical analysis features. We have maintained two clear visions of e-rater’s development. First, new linguistically-based ...
متن کاملSurprisal as a Predictor of Essay Quality
Modern automated essay scoring systems rely on identifying linguistically-relevant features to estimate essay quality. This paper attempts to bridge work in psycholinguistics and natural language processing by proposing sentence processing complexity as a feature for automated essay scoring, in the context of English as a Foreign Language (EFL). To quantify processing complexity we used a psych...
متن کامل